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The NASA STI Program Office ... in Profile 


Since its founding, NASA has been dedicated to the 
advancement of aeronautics and space science. The 
NASA Scientific and Technical Information (STI) 
Program Office plays a key part in helping NASA 
maintain this important role. 

The NASA STI Program Office is operated by 
Langley Research Center, the lead center for NASA’s 
scientific and technical information. The NASA STI 
Program Office provides access to the NASA STI 
Database, the largest collection of aeronautical and 
space science STI in the world. The Program Office is 
also NASA’s institutional mechanism for 
disseminating the results of its research and 
development activities. These results are published by 
NASA in the NASA STI Report Series, which 
includes the following report types: 

• TECHNICAL PUBLICATION. Reports of 
completed research or a major significant phase 
of research that present the results of NASA 
programs and include extensive data or 
theoretical analysis. Includes compilations of 
significant scientific and technical data and 
information deemed to be of continuing 
reference value. NASA counterpart of peer- 
reviewed formal professional papers, but having 
less stringent limitations on manuscript length 
and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific 
and technical findings that are preliminary or of 
specialized interest, e.g., quick release reports, 
working papers, and bibliographies that contain 
minimal annotation. Does not contain extensive 
analysis. 

• CONTRACTOR REPORT. Scientific and 
technical findings by NASA-sponsored 
contractors and grantees. 


• CONFERENCE PUBLICATION. Collected 
papers from scientific and technical 
conferences, symposia, seminars, or other 
meetings sponsored or co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 
technical, or historical information from NASA 
programs, projects, and missions, often 
concerned with subjects having substantial 
public interest. 

• TECHNICAL TRANSLATION. English- 
language translations of foreign scientific and 
technical material pertinent to NASA’s mission. 

Specialized services that complement the STI 
Program Office’s diverse offerings include creating 
custom thesauri, building customized databases, 
organizing and publishing research results ... even 
providing videos. 

For more information about the NASA STI Program 
Office, see the following: 

• Access the NASA STI Program Home Page at 
http://www.sti.nasa.gov 

• E-mail your question via the Internet to 
help@sti.nasa.gov 

• Fax your question to the NASA STI Help Desk 
at (301) 621-0134 

• Phone the NASA STI Help Desk at 
(301) 621-0390 

• Write to: 

NASA STI Help Desk 

NASA Center for AeroSpace Information 

7121 Standard Drive 

Hanover, MD 21076-1320 
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Summary 


The NASA Science Files™ (also the NASA SCI Files™) <http://scifiles.larc.nasa.gov> is an Emmy®- 
award-winning series of instructional programs for grades 3-5. Produced by the NASA Center for Dis- 
tance Learning <http://dlcenter.larc.nasa.gov>, programs in the series are research-, inquiry-, standards-, 
teacher-, and technology-based. Each NASA SCI Files™ program (1) integrates mathematics, science, 
and technology; (2) uses Problem-Based Learning (PBL) to enhance and enrich the teaching and learning 
of science; (3) emphasizes science as inquiry and the scientific method; (4) motivates students to become 
critical thinkers and active problem solvers; and (5) uses NASA research, facilities, and personnel to raise 
student awareness of careers in math, science, and technology and to exhibit the “real world” application 
of mathematics, science, and technology. On April 15, 2004, 1,500 randomly selected registered users of 
the NASA SCI Files™ were invited to complete an electronic (self-reported) survey containing a series of 
questions, grouped in eight categories that employ a 5-point Likert-type scale response. This report con- 
tains the quantitative and qualitative results of that survey. In all, 263 surveys (17.6 percent response rate) 
were received by the established cut-off date, May 15, 2004. Demographically, around 79 percent of the 
respondents were female, just about 8 1 percent identified their current professional duties as “classroom 
teacher,” almost 75 percent worked in a public school, and nearly 56 percent held a master’s degree or 
master’s equivalency. Respondents reported that the NASA SCIence Files™ programs (1) are a valuable 
instructional aid ( x = 4.57); (2) are aligned with the national mathematics, science, and technology stan- 
dards (x = 4.50); (3) raise student awareness of careers requiring mathematics, science, and technology 
(x = 4.30); (4) demonstrate the application of mathematics, science, and technology (x = 4.35); and 
(5) increase student motivation and enthusiasm for learning (x = 4.50). 

Introduction 

The NASA Center for Distance Learning (CDL) is recognized for (1) its leadership in the application 
of traditional and emerging instructional technology; (2) the development of six exciting, innovative, and 
inspirational instructional and educational programs that are an integral part of NASA’s Integrated Dis- 
tance Learning Network; (3) its use of NASA programs, projects, facilities, and personnel to motivate and 
inspire teaching and learning; and (4) its ability to identify customer needs and to translate those needs 
into customer-focused programs. Originating as a collaboration with Christopher Newport University in 
1996, the six programs offered by the NASA CDL “span the educational horizon” from grades K-12, 
through college (grades 13—1 8), to adult (lifelong) learners. 

The Emmy®-award-winning programs produced by the NASA CDL are research-, inquiry-, 
standards-, and teacher-based. They are technology-focused programs that (1) promote creativity, critical 
thinking, and problem-solving skills; (2) integrate easily, in whole or in part, into an existing curriculum, 
and can introduce or reinforce a curriculum topic, objective, or skill; (3) serve both formal and informal 
education; (4) increase interest, engagement, and understanding of science, technology, engineering, and 
mathematics (STEM); (5) motivate and inspire students to pursue careers in STEM areas; (6) establish a 
connection between STEM concepts taught in the classroom and those used every day by NASA 
researchers; (7) are readily accessible to homebound and home schooled children; (8) increase (adult) 
scientific and technological literacy; (9) are closed- and (audio) descriptive-captioned and 508 compliant; 
(10) use technology to enhance and enrich the teaching and learning process; (11) advance the theory and 
practice of teaching mathematics, science, and technology; (12) support the NASA education strategy; 
(13) contribute to the nation’s science and engineering goals; (14) support the Agency’s workforce devel- 
opment initiatives; and (15) communicate the results of NASA discovery, exploration, innovation, and 
research. 



Science for All Americans: Project 2061 states that children can learn most readily about things that 
are tangible and directly accessible to their senses and that constructive, concrete experiences are most 
effective when they occur in the context of some relevant conceptual structure (AAAS, 1989). Children 
are naturally curious and want to know and understand the “why” of their world (National Research 
Council, 2000). Although investigation of the natural world may take a variety of forms, the NASA 
SCI Files™ uses inquiry as a strategy (1) that builds on children’s natural inquisitiveness; (2) that helps 
students understand mathematics, science, and technology as human endeavors; (3) that assists students in 
acquiring knowledge, developing problem-solving and critical-thinking skills, and fostering creativity; 
and (4) that introduces students to mathematics, science, and technology career fields. The series uses 
Problem Based Learning (PBL), a form of inquiry-based teaching, which allows students to take an active 
role in the learning process. PBL empowers students with the responsibility of managing a largely self- 
directed learning process (Boud and Felietti, 1997). PBL also encourages students to develop skills that 
will enable them to understand the relationships between mathematics, science, and technology and to 
become adult (lifelong) learners (Brine and Shannon, 1997). Coupled with inquiry -based learning, the 
NASA SCIence Files™ uses PBL and “real world” problems to make learning mathematics, science, and 
technology active, interesting, and relevant to students (Cawelti, 1999). 

Overview of NASA SCIence Files™ 

Officially released in 1999 and titled the NASA “Why?” Files, the NASA SCIence Files™ is the sec- 
ond oldest program produced by the NASA Center for Distance Learning. Each program in the series has 
the following three components: a 60-minute television broadcast (divided into four 15-minute teachable 
segments), an educator guide that contains hands-on activities, and an interactive PBL web activity that 
provides educators an opportunity to integrate technology into the classroom setting, thus enabling stu- 
dents to further explore topics presented in the broadcast. The NASA SCIence Files”' and the NASA SCI 
Files™ are trademarks owned by the National Aeronautics and Space Administration (NASA). (Visit 
<http://scifiles.larc.nasa.gov> for more information.) 

The Society of Women Engineers (SWE), the NASA SCI Files™ professional engineering collabora- 
tor, provides registered users with classroom mentors. SWE, the largest nonprofit educational and service 
organization for women engineers, represents both student and professional women in engineering and 
technical fields. The society has over 17,000 members and more than 400 members at large. SWE’s pri- 
mary objectives include informing young women, their parents, counselors, and the general public of the 
qualifications and achievements of women engineers and the opportunities open to them. SWE also 
stimulates women’s interest in achieving full potential in careers as engineers and leaders. Hampton (VA) 
City Public Schools (HCPS) is the NASA SCI Files™ education collaborator. HCPS educators and other 
educators develop the classroom and web-based activities for each program. Busch Gardens Williams- 
burg (VA) is our corporate collaborator. They provide talent fees and location shoots for the tree house 
detectives. 

For additional information regarding these collaborations, contact NASA Langley’s Center for Dis- 
tance Learning via email (dlcenter+mail@larc.nasa.gov). Information may also be obtained by contacting 
SWE (Karen.Horting@SWE.org), Busch Gardens (Kimberly.Laska@BuschGardens.com), or Hampton 
City Public Schools (PJohnson@SBO.Hampton.K12.va.us). 

The NASA SCIence Files™ series has received numerous awards for program achievement, educa- 
tional content, web site content, and technical production. Find a complete list of the NASA SCI Files™ 
awards at <http://scifiles.larc.nasa.gov/text/awards.html>. 
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In the 2003-2004 program season, the NASA SCIence Files™ series won an Emmy® for Outstanding 
Youth Programming, and The Case of the Prize Winning Plants received an Aegis award in the categoiy 
“education/science.” 

NASA is the copyright owner for all the NASA SCIence Files™ programs produced since March 
2002. Although copyrighted, NASA grants to users (e.g., formal and informal educators) and television 
stations an unlimited non-exclusive license to use, reproduce, and perform and display publicly the copy- 
righted works, with the proviso that users and television stations register with the NASA CDL. Users can 
register in one of four ways for the NASA SCI Files™ : 

(1) e-mail <dlcenter+mail@larc.nasa.gov> 

(2) online <http://scifdes.larc.nasa.gov> 

(3) telephone 757-864-6100 

(4) USPS: NASA SCI Files™ 

Office of Communications and Education 
Mail Stop 400-DL 
NASA Langley Research Center 
Hampton, VA 23681-2199 

Users registering as educators must specify the number of students viewing each program and televi- 
sion stations must specify the potential (viewing) audience. Educators are granted unlimited rights for 
duplication, dubbing, broadcasting, cable casting, and web casting when the NASA SCI Files™ materials 
are used for educational purposes. No fees or licensing agreements are required for registered users of 
programs in this series. Programs in the NASA SCI Files™ series may not be used, either in whole or in 
part, for commercial purposes without the express written permission (i.e., consent) of NASA. 

As of September 30, 2004, 145,600 (formal and informal) educators, representing 4.3 million students 
and 384 television stations, with a combined (potential) audience of 157.6 million, were registered users 
of the NASA SCI Files™. Programs in the NASA SCIence Files™ series are uplinked (via satellite) in 
both KU- and C-band. Each program complies with the specifications found in the National Educational 
Telecommunications Association (NETA) Common-Sense Guide to Technical Excellence. Programs 
( 1 ) air nationally on Cable Access, ITV (instructional television), NASA TV, and Public Broad- 
casting System (PBS)-member stations; (2) can be streamed from the Apple Learning Interchange 
(ALI) <ali. apple. com>, <www.knowitall.org>, and ibiblio at the University of North Carolina 
<nasa.ibiblio.org/connect.php>; (3) air on state-wide educational television systems such as South 
Carolina Eduational Television (SC ETV) and district wide educational television systems such as 
Virginia Beach Television (VBTV); (4) can be obtained from the NASA Education Resource Centers 
(ERCs) <www.nasa.gov>; and (5) can be purchased from NASA CORE (Central Operation of Resources 
for Educators) <http://core.nasa.gov>. There are nine programs in the 2003-2004 NASA SCIence Files™ 
broadcast season; four were new programs and five were repeat programs. 

Evaluation 

We use evaluation to obtain objective information that can help us determine the success of our dis- 
tance learning programs and provide information for continuous improvement. For us, evaluation is an 
ongoing process that provides accurate and reliable information. We use evaluation (1) to approximate the 
cost/benefit of our programs; (2) as an accountability tool; (3) to help make sound decisions relating to 
program design, personnel, and budget; and (4) to determine whether our program objectives are met. We 
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use various tools to help us obtain objective data. In addition to the NASA Educational Evaluation Infor- 
mation System (NEEIS), we use (1) focus group interviews, (2) telephone surveys, (3) mail and electronic 
(self-reported) surveys, and (4) market research to collect qualitative and quantitative data from two 
groups: intermediaries (television station managers that represent stations airing the NASA SCI Files™) 
and consumers (formal and informal educators) who are registered users of our programs. 

In addition to direct program evaluation, we have developed a series of metrics to measure the success 
of our marketing efforts and the overall quality of our programs. Key metrics for 2003-2004 include web 
site visitors (33,410 monthly), unique web site visitors (17,114 monthly), educator guide downloads 
(52,172 per annum), number of registered users (145,576), number of registered stations (384), number of 
video copies sold by NASA CORE (2,321 per annum), and PBS market coverage (32 percent). Web- 
based metrics are determined by using Web Trends®, an industry standard web statistics package. To 
determine metrics related to public television, we license and use an external database managed by 
PubTV . By licensing from an outside source specializing in PBS data, we can ensure the validity of the 
information and access metrics such as air times that we could not develop in-house. We cull statistics 
that focus on registered users from our in-house registration database. In addition to providing a form of 
measurement, this database provides a way to directly contact individual users and allows us to conduct 
the telephone and mail surveys discussed previously. (See, for example, Pinelli and Perry, May 2004.) 
Overall, these metrics indicate both awareness and use of our programs by key market segments. The 
metrics, including a summary, are updated each month. Metrics for the 1 st , 2 nd , 3 rd , and 4 th quarters of 
Fiscal Year 2004 are located on our web site <http://dlcenter.larc.nasa.gov/reports> and can be accessed 
only from within the NASA Langley Research Center network. 

Methodology 

We randomly drew a sample of 1,500 registered users from the NASA SCI Files™ database, contacted 
them by email, and asked them to participate in the program evaluations of the 2003-2004 broadcast 
season. Each member in the sample group received a link to an electronic (self-reported) survey/ 
questionnaire in early April 2004. The survey contained 88 questions, 10 of which dealt with demo- 
graphics. Respondents had the option of requesting a free copy of the final assessment report, and all 
users who completed and submitted (electronically) a survey received a NASA educational compact disk 
(CD). We received 263 usable surveys by May 15, 2004, the established cut-off date. The overall 
response rate for the 2003-2004 NASA SCI Files™ evaluation project was 17.6 percent. 

Organization of Report 

The report begins with a summary followed by an introduction, overview, demographics, presentation 
of the qualitative and quantitative data, the interpretation of the data, concluding remarks (including rec- 
ommended changes and/or topics and/or areas for further evaluation), and references. Appendix A 
contains a list of the programs, by title and description, in the NASA SCI Files™ 2003-2004 broadcast 
season, appendix B is the online survey, and appendix C contains the qualitative data. The qualitative data 
come from the evaluation questions that allowed respondents to offer “other” as a response and/or to 
qualify their response. We also incorporated the qualitative data we collected into the suggested changes 
for the 2004-2005 NASA SCI Files™ season. This report is available on the Langley Technical Reports 
Server (LTRS), <http://techreports.larc.nasa.gov/ltrs/ltrs.html>. 
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Demographics 


We asked survey participants a series of demographic questions, the answers to which enabled us to 
establish the following respondent profile (findings) for the NASA SCI Files™ 2003-2004 survey. 

• About 79 percent of the respondents were female. 

• About 46 percent of the respondents were located in suburban school districts, 27 percent in rural 
school districts, and 26 percent in urban school districts. 

• About 80 percent of the respondents were classroom teachers. 

• About 75 percent of the respondents worked in a public school. 

• About 56 percent of the respondents held a master’s degree or master’s equivalency. 

• About 87 percent of the respondents were self-identified as Caucasian. 

• Most of the respondents were in their forties. 

• The mean and median “years as a professional educator” were 12.98 and 1 1.0, respectively. 

• About 98 percent of the respondents owned a personal computer. 

Presentation of Quantitative Data 

Survey questions pertain to seven topics. In this section, the aggregate data for each question are pre- 
sented for each of the seven topics. (Responses to the demographics were previously presented.) The data 
are reported in terms of the mean (average) when the question used a 5-point Likert (response) scale and 
in percentages when the question required other responses. Mean values appear in parentheses following 
appropriate questions. The statistical values for responses on a 5-point Likert scale to each question were 
calculated by using the number of respondents that answered a particular question (n) rather than by the 
number from the total population of respondents (N). 

Topic 1. Instructional Technology and Teaching 

Respondents were asked to rate nine statements related to instructional technology and teaching 
(table 1). The highest mean rating (x = 4.48) was given to the statement that instructional technology 
increases student motivation and enthusiasm for learning. The next highest mean ratings were given 
to the statements that instructional technology helps teachers accommodate different learning styles 
(x = 4.46), technology enables teachers to be more creative (x = 4.38), and technology enables teachers 
to teach more effectively (x = 4.38). At slightly lower mean ratings, the respondents reported that 
instructional technology increases student learning comprehension (x = 4.26) and that, in general, 
instructional programs they have seen are of good quality (x = 3.92). The lowest mean rating (x =3.21) 
was given to the statement that teachers are generally eager to use instructional technology in the class- 
room. 

Respondents also received a list of five factors that could prohibit or limit their use of instructional 
technology in teaching. They were asked to indicate which of these factors they considered as barriers 
that keep them from using instructional technology in their teaching (fig. 1). Respondents were not 
limited to selecting one factor; instead, they could select all factors that applied. Respondents (n = 262) 
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indicated that not enough or limited access (58 percent) was the largest barrier, followed by lack of time 
(47 percent), not enough software (39 percent), and lack of technical support (26 percent). Fourteen 
percent of respondents indicated that there were no barriers that kept them from using instructional tech- 
nology in their teaching. 


Table 1. Instructional Technology and Teaching 


Questions 1-9 

Mean 

Median 

Standard 

deviation 

Min. 

Max. 

Responses 

(n) 

Instructional technology helps teachers teach 
more effectively. 

4.38 

5 

0.89 

1 

5 

260 

Instructional technology helps teachers 
accommodate different learning styles. 

4.46 

5 

0.82 

1 

5 

261 

Instructional technology helps teachers be 
more creative. 

4.38 

5 

0.90 

1 

5 

259 

Instructional technology improves student 
learning comprehension. 

4.26 

4 

0.88 

1 

5 

262 

Instructional technology increases student 
motivation and enthusiasm for learning. 

4.48 

5 

0.85 

1 

5 

258 

In my experience, administrators support and 
encourage teachers to use instructional 
technology in the classroom. 

3.89 

4 

1.08 

1 

5 

262 

Teachers are generally eager to use 
instructional technology in the classroom. 

3.21 

3 

0.92 

1 

5 

263 

In general, the instructional programs I’ve 
seen are of good quality. 

3.92 

4 

0.90 

1 

5 

261 

The technology training provided by my 
school division has improved my computer 
skills. 

3.54 

4 

1.17 

1 

5 

259 


(n) denotes number of responses. 

A 1-5 point scale (5 indicating “strongly agree”) was used to measure agreement. 
Min. denotes minimum. 

Max. denotes maximum. 


Q 10. Are there any barriers that keep you from using more 
instructional technology in your teaching? (n = 262) 
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Figure 1. Barriers to using instructional technology in teaching. 
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Topic 2. NASA Science Files Television/Video Programs 

Use of Television/Video Programs 

Respondents were asked if they used any of the nine NASA SCIence Files™ 2003-2004 programs 
(fig. 2). Of the respondents, 117 indicated that they had used the programs, 0 indicated that they had not, 
and 1 indicated that he or she might in the future. 


Q 11. Have you used any of the 9 NASA 
Science Files™ 2003-2004 programs? (n = 118) 
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Figure 2. Use of programs in the NASA SCIence Files™ series. 


Respondents were then asked specifically which NASA SCI Files™ programs (1 through 9) their stu- 
dents viewed (fig. 3). 


I I I I l, K I I I I 

I I l| I I. I l| I I 

1 I II la II I. ll I I 


Figure 3. Use of programs in the NASA SCIence Files™ series. 

Respondents (n = 119) were asked to indicate how many years they have used the NASA SCI Files™ 
programs. Fifty (42 percent) had used the programs for 1 year, 30 (25 percent) had used the programs 
2 years, 21(18 percent) used the programs for 3 years, 12 (10 percent) used the programs for 4 years, and 
6 (5 percent) used the programs for 5 years. 
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Respondents who used the NASA SCI Files”' programs were asked to identify how they used them 
in their classes (fig. 4). 
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Figure 4. How NASA SCIence Files”' programs are used in the classroom. 


Grade Levels Using Television/Video Programs 

Respondents who used the 2003-2004 NASA SCIence Files”' were asked to report which grade levels 
viewed the programs (fig. 5). 


Q 12. What grade level(s) viewed the NASA SCIence Files™ programs? 



Figure 5. Grade levels viewing NASA SCIence Files™ programs. 


Acquiring Television/Video Programs 

How did respondents receive the programs? One hundred sixteen responded to this question. Eighteen 
(16 percent) reported they received the programs via PBS, 29 (25 percent) said that they downloaded 
(downlinked) the programs, 15(13 percent) stated that a media specialist taped the programs for them, 
18 (16 percent) indicated that they taped (or had someone else tape) the programs, and finally, 36 
(3 1 percent) stated that they received the programs from a NASA Educator Resource Center (ERC). 

Ease of Attaining Television/Video Programs 

A follow-up question asked whether respondents experienced any difficulty obtaining any of the 
programs in the 2003-2004 series. Of the 115 respondents, 39 percent indicated experiencing difficulty 
obtaining the programs; down slightly from 41 percent in the 2002-2003 season. 

Quality of NASA Science Files ™ Television/Video Programs 

Respondents rated the overall quality of the NASA SCI Files™ programs (fig. 6) and the quality of the 
videos in the NASA SCIence Files 2003-2004 series (fig. 7). Eighty-six percent of the respondents 
(n = 116) rated the overall quality of the NASA SCIence Files™ programs better than average, 14 percent 
rated the quality average, and no one rated the quality worse than average. For the instructional videos, 
78 percent of the respondents (n = 116) rated the videos better than average, 21 percent rated the videos 
average, and 1 percent rated the videos worse than average. 


Q 13. Compared to other instructional 
programming, is the OVERALL quality of 
NASA SCIence Liles™: (n = 116) 
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Figure 6. Quality of NASA SCIence Files™ programs. 
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Compared to the VIDEO in other instructional programs, 
were the NASA SCI Files™ videos: (n = 116) 
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Figure 7. Quality of video in NASA SCIence Files™ programs. 

The last component of the NASA SCI Files™ television/video program evaluation process asked 
respondents to assess program content and quality by indicating their level of agreement with 15 state- 
ments (table 2). The statements receiving the strongest support from the respondents were that the 
programs made learning science and math interesting (x = 4.59) and that the programs were a valuable 
instructional aid (x = 4.57). High marks also went to the statements that the programs were of good 
technical quality (x = 4.53), that the programs increased student motivation and enthusiasm for learning 
(x = 4.50), and that the programs were aligned with national mathematics, science, and technology stan- 
dards (x = 4.50). The lowest scores were attributed to these statements: “the programs raised student 
awareness of careers that require mathematics, science, and technology” (x = 4.30) and “the programs 
demonstrated the application of mathematics, science, and technology on the job” (x = 4.35). 
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Table 2. Quality of NASA SCI Files™ Tele vision/ Video Programs 


Question: Please indicate the extent to which 
you disagree or agree with the following 
statements concerning the four programs in the 
2003-2004 NASA SCIence Files™ series. 

Mean 

Median 

Standard 

deviation 

Min. 

Max. 

Responses 

(n) 

The programs were well organized. 

4.43 

5 

0.66 

2 

5 

117 

The programs were of good artistic quality. 

4.39 

5 

0.78 

1 

5 

117 

The programs were of good technical quality. 

4.53 

5 

0.65 

1 

5 

117 

The programs made learning science and math 
interesting. 

4.59 

5 

0.68 

1 

5 

115 

The programs helped me accommodate 
different learning styles. 

4.37 

5 

0.80 

1 

5 

116 

The programs increased my students’ 
knowledge of science and math. 

4.39 

5 

0.82 

1 

5 

117 

The programs increased student motivation 
and enthusiasm for learning. 

4.50 

5 

0.72 

1 

5 

116 

The programs were a valuable instructional 
aid. 

4.57 

5 

0.67 

1 

5 

117 

The programs were appropriate for the 
specified grade level. 

4.37 

5 

0.79 

2 

5 

115 

The programs were easily incorporated into the 
curriculum. 

4.44 

5 

0.64 

2 

5 

116 

The programs enhanced the integration of 
mathematics, science, and technology. 

4.48 

5 

0.69 

2 

5 

117 

The programs raised student awareness of 
careers that require mathematics, science, and 
technology. 

4.30 

5 

0.89 

1 

5 

117 

The programs demonstrated the application of 
mathematics, science, and technology on the 
job. 

4.35 

5 

0.78 

1 

5 

116 

The programs were aligned with national 
mathematics, science, and technology 
standards. 

4.50 

5 

0.75 

1 

5 

116 

The programs presented females and 
minorities performing challenging engineering 
and scientific tasks. 

4.43 

5 

0.77 

1 

5 

117 


(n) denotes number of responses. 

A 1-5 point scale (5 indicating “strongly agree”) was used to measure agreement. 
Min. denotes minimum. 

Max. denotes maximum. 
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Length of Television/Video Programs 

Each program in the NASA SCI Files™ series is 60 minutes long. Respondents were asked to give 
their opinion as to the length of the 2003-2004 NASA SCI Files™ programs (fig. 8). 


Q 46: Is the 60-minute length of the programs... 
(n = 116) 
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Figure 8. Length of NASA SCI Files™ Program. 

Topic 3. NASA Science Files™ Educator Guides 

Use of Educator Guides 

Note that the terms educator guide and lesson guide are used interchangeably throughout this report. 
Respondents were asked if they used the NASA SCI Files”' educator guides. Of the respondents, 101 
(86 percent) indicated that they had used the educator guides, 16 (14 percent) indicated that they had not, 
and 1 (<1 percent) indicated that he or she might in the future (fig. 9). 
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Q 48: Did you use the lesson guides for any of the 
2003-2004 NASA Science Files™ programs? (n = 118) 
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Figure 9. Use of NASA SCI Files™ Educator Guides. 
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Quality of Educator Guides 


Respondents were asked their opinion of the NASA Science Files™ educator guides as compared 
with other educator guides. Of the respondents, 101 indicated that the educator guides were “better than 
average,” 16 indicated that the educator guides were “about average,” and 7 indicated that they did not 
review the guides (fig. 10). 


Q 49: Compared to other LESSON GUIDES, were 
NASA Science Files™ lesson guides: (n = 124) 
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Figure 10. Quality of NASA SCI Files™ educator guides. 

The respondents were then asked to react to six statements about the quality of the NASA SCIence 
Files™ educator guides (table 3). Respondents indicated that the educator guides correlated well with the 
video, giving it the highest mean rating (x = 4.55), followed by the statement that the print and electronic 
resources in the lesson guides were valuable, (3c = 4.54). High scores also went to the statement that the 
layout of the educator guides presented information clearly (3c = 4.52). The statement that the educator 
guides were easily downloaded from the Internet received the lowest mean rating (3c = 4.36). 


Table 3. Quality of NASA SCIence Files™ Educator Guides 


Questions 

Mean 

Median 

Standard 

deviation 

Min. 

Max. 

Responses 

(n) 

The lesson guides were a valuable instructional 
aid. 

4.43 

5 

0.72 

2 

5 

119 

The lesson guides were easy to download from 
the Internet. 

4.36 

5 

0.84 

2 

5 

119 

The lesson guides correlated well with the video. 

4.55 

5 

0.67 

1 

5 

116 

The directions/instructions in the lesson guides 
were easy to understand. 

4.48 

5 

0.69 

1 

5 

118 

The layout of the lesson guides presented the 
information clearly. 

4.52 

5 

0.66 

2 

5 

118 

The print and electronic resources in the lesson 
guides were valuable to me. 

4.54 

5 

0.69 

2 

5 

119 


(n) denotes number of responses. 

A 1-5 point scale (5 indicating “strongly agree”) was used to measure agreement. 
Min. denotes minimum. 

Max. denotes maximum. 


13 


Topic 4. NASA Science Files™ Problem-Based Learning Activities 
Use of Problem-Based Learning Activities 

Respondents were asked whether their students used any of the PBL activities found on the NASA 
SCIence Files™ web site. Of the respondents, 50 indicated that they had, 75 indicated that they had not, 
and 138 indicated that they may in the future (fig. 1 1). 


Q 57: Did your students use any of the PBL activities found on the 
NASA SCIence Files™ web site? (n = 163) 
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Figure 11. Use of NASA SCI Files™ PBL activities. 


Quality of PBL Activities 

Respondents were asked their opinion of the NASA SCIence Files™ PBL activities as compared with 
other PBL activities. Of the respondents, 36 (71 percent) indicated that the activities were “better than 
average” and 15 (29 percent) indicated that the activities were “about average.” No respondents indicated 
that the activities were “worse than average” or that they did not review the web-based activities (fig. 12). 


Q 58: Compared to other PBL activities, was the quality of the PBL activities 
for the NASA SCIence Files™: (n = 51) 
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Figure 12. Quality of NASA SCI Files™ PBL activities. 
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Next, respondents were asked to indicate the extent to which they agreed with the following state- 
ments concerning the quality of the PBL activities posted on the NASA SCI Files™ web site (table 4). 
Respondents gave the highest mean rating to the statements “the PBL activities were a valuable instruc- 
tional aid” (3c = 4.48) and “the PBL activities enhanced the integration of mathematics, science, and 
technology” (3c = 4.48). High scores also went to the statements “the PBL activities were easily inte- 
grated into the curriculum” ( x = 4.32) and “the PBL activities raised student awareness of careers that 
require mathematical, technological, and scientific knowledge” (3c = 4.30). Respondents gave the lowest 
mean rating to the statement “students were able to complete the PBL activities in a reasonable amount of 
time” (3c = 4.06). 


Table 4. NASA SCI Files™ Problem-Based Learning Activities 


Question: Please indicate the extent to which 
you disagree or agree with the following 
statements concerning the problem-based 
learning activity posted on the NASA SCI 
Files™ web site. 

Mean 

Median 

Standard 

deviation 

Min. 

Max. 

Responses 

(n) 

The PBL activities are a valuable instructional 
aid. 

4.48 

5 

0.64 

3 

5 

50 

The PBL activities were easily integrated into 
the curriculum. 

4.32 

4 

0.71 

3 

5 

50 

The PBL activities raised student awareness 
of careers that require mathematical, 
technological, and scientific knowledge. 

4.30 

4 

0.79 

2 

5 

49 

Students were able to complete the PBL 
activities in a reasonable amount of time. 

4.06 

4 

0.82 

3 

5 

49 

The PBL activities accommodated various 
learning styles. 

4.31 

4 

0.74 

2 

5 

48 

The content of the PBL activities was 
appropriate for my students. 

4.22 

4 

0.70 

3 

5 

50 

The PBL activities enhanced the integration 
of mathematics, science, and technology. 

4.48 

5 

0.64 

3 

5 

50 


(n) denotes number of responses. 

A 1-5 point scale (5 indicating “strongly agree”) was used to measure agreement. 
Min. denotes minimum. 

Max denotes maximum. 
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Topic 5. NASA Science Files™ Web Site 


Respondents were asked whether they visited the NASA SCI Files™ web site (fig. 13). One-hundred 
percent of the 203 respondents indicated that they visited the web site. 



Respondents were asked to indicate the extent to which they agreed with the following statements 
concerning the 2003-2004 NASA SCI Files™ web site (table 5). Respondents gave the highest mean 
ratings to the statements “the NASA SCI Files™ web site is visually appealing” ( x = 4.48), and “the 
external links are good resources for further exploration” (x = 4.43). High ratings were also given to the 
statement “the web site complements the broadcast/video” (x = 4.36). Respondents gave the lowest 
mean rating to the statement “the NASA SCI Files™ web site is easy to navigate” (x = 4.23). 

Table 5. Quality of NASA SCI Files™ Web Site 


Question: Indicate the extent to 
which you agree/disagree with the 
following statements. 

Mean 

Median 

Standard 

deviation 

Min. 

Max. 

Responses 

(n) 

The NASA SCI Files™ web site is 
visually appealing. 

4.48 

5 

0.70 

1 

5 

201 

The NASA SCI Files™ web site is 
easy to navigate. 

4.23 

4 

0.85 

1 

5 

201 

The links to other sites/pages are 
current/up to date. 

4.33 

4 

0.79 

1 

5 

190 

The external links are good 
resources for further exploration. 

4.43 

5 

0.70 

1 

5 

188 

The web site complements the 
broadcast/video. 

4.36 

5 

0.77 

2 

5 

150 


(n) denotes number of responses. 

A 1-5 point scale (5 indicating “strongly agree”) was used to measure agreement. 
Min. denotes minimum. 

Max. denotes maximum. 


16 


Topic 6. Overall Assessment of NASA SCI Files™ 

Respondents were asked whether they would recommend the NASA Science Files™ to a colleague if 
that colleague inquired (fig. 14). Of the respondents, 254 (98 percent) indicated that they would recom- 
mend NASA Science Files”’ and 4 (2 percent) indicated that they would not. 


Q 73. If a colleague inquired, would you recommend the 
NASA Science Files™ to him/her? (n = 258) 
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Figure 14. Recommending NASA SCIence Files™ . 

Respondents were asked whether they think the NASA SCIence Files”’ has been successful in fulfill- 
ing its goal of educating others about what NASA does (fig. 15). Of the respondents, 249 (97 percent) 
indicated “yes,” they feel NASA has been successful in this regard, while 9 (3 percent) respondents indi- 
cated “no,” they do not feel NASA has been successful in this regard. 


Q 74. One of our goals is to educate and inform others 
about what NASA does. Do you think NASA SCIence 
Files™ has been successful in this regard? (n = 258) 
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Figure 15. Success ofNASA SCI Files™ programs. 
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Respondents were asked the degree to which they felt that the information about NASA contained in 
the programs was credible (fig. 16). Of those responding, 226 (95 percent) indicated that the information 
was “very credible” and 12 (5 percent) indicated that the information was “somewhat credible.” No 
respondents indicated that the information was “not credible” or that they were unable to judge. 


Q 75. In your opinion, was the information about NASA 
contained in the programs: (n = 238) 
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Figure 16. Credibility of infonnation contained in NASA SCI Files™ . 

Topic 7, Videoconferencing 

Respondents were asked whether they had access to videoconferencing equipment for instructional 
purposes (fig. 17). Of those responding, 36 (97 percent) indicated that they did have access while 1 
(3 percent) indicated they did not. 


Q 76. Do you have access to 
videoconferencing equipment for your 
instruction? (n = 37) 
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Figure 17. Videoconferencing capability. 
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Respondents were then asked whether they would be interested in having their students participate 
in a NASA-sponsored videoconference in the future (fig. 18). Of the respondents, 34 indicated that they 
would be interested in videoconferencing. No respondents indicated that they did not want to participate 
in a NASA-sponsored videoconference. 


Q 77. In the future, would you be interested 
in having students participate in a 
NASA-sponsored videoconference? (n = 34) 
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Figure 18. Interest in participating in NASA-sponsored videoconference. 

Lastly, respondents were asked what their language preference would be for a NASA-sponsored 
videoconference (fig. 19). Of the respondents, 32 (89 percent) indicated a preference for English, 1 
(3 percent) indicated a preference for Spanish, and 3 (8 percent) indicated a preference for both English 
and Spanish. 


Q 78. For a NASA-sponsored videoconference, what would be 
your language preference? (n = 36) 
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Figure 19. Language preference for NASA-sponsored videoconference. 
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Presentation of Qualitative Data 


The qualitative data come from the evaluation questions that allowed respondents to offer “other” as a 
response and/or to qualify their responses. Almost 100 qualitative responses were submitted (see appen- 
dix C). The majority of the responses were “positive comments” that related to the educational value and 
technical quality of the programs. 

1. Several respondents reported using only one or more program components (e.g., the experiments 
and activities sheets). Lack of “available classroom time” was the reason cited most often for their 
decision. The most commonly reported reasons were “I wish I had more time to fully use the 
programs” and “our district/school prescribes what can/cannot be used in the classroom, regardless 
how useful/valuable that resource might be.” 

2. Several respondents indicated they had difficulty acquiring/obtaining the programs. “Our PBS 
station does not air the NASA programs” was most frequently reported. Several homeschoolers 
reported having problems acquiring the programs. One respondent complained (bitterly) about the 
difficulty encountered obtaining the NASA SCI Files™ programs from a NASA ERC. 

3. Technical difficulties encountered ranged from downlinking the signal to figuring out how to “set” 
the VCR to record the programs. At least two respondents lamented that the NASA CDL no longer 
mailed (printed copies) of the educator guides. 

4. Several respondents reported that the “background music” was too jumpy, that the music 
overwhelmed the spoken word, and that the “actors” talked too fast; therefore, the students were 
not able to focus/concentrate on the “actual concept” being presented. 

5. Several respondents reported using the Spanish-language” versions of the NASA SCI Files™ and 
indicated that they would use the PBL activities (if available in Spanish). 

Interpreting the Data 

Having presented the survey findings in the previous section, the next step is to interpret them in terms 
of assessing the programs in the NASA SCIence Files™ 2003-2004 broadcast season and comparing 
them to previous seasons. Excluding the survey demographics, interpretations and comparisons of the 
findings for the remaining seven survey topics are presented. Where appropriate, comparisons are made to 
the data reported in the evaluation of the NASA SCIence Files™ programs for previous broadcast 
seasons. (See Pinelli, 2002 and 2004). The survey did change appreciably in the 2003-2004 season, and 
we only make comparisons where appropriate. Note that some of the wordings changed slightly from 
previous years, and between 2001-2002 and 2002-2003, the extremes of the 5-point Likert scale went 
from disagree/agree to strongly disagree/strongly agree. 
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Topic 1. Instructional Technology and Classroom Teaching 


Based on the data in table 6, it is apparent that those surveyed over the years believe that instructional 
technology increases student motivation and enthusiasm for learning and assists in accommodating the 
different learning styles of students (both have the highest average means of 4.49 shown in the right-hand 
column under “average”). Those surveyed also believe that the use of instructional technology helps 
teachers to be more creative (average mean = 4.46), helps teachers teach more effectively (average 
mean = 4.43), and to a lesser extent improves students’ learning comprehension (average mean = 4.32). In 
short, the educators who participated in the survey believe in the promise of instructional technology to 
enhance and enrich the teaching and learning process. 

Table 6. Instructional Technology and Teaching Comparison Means 


Question 

2000-200 If 

2001-2002f 

2002-2003 

2003-2004 

Average 

Instructional technology helps* teachers 
teach more effectively. 

4.42 

4.61 

4.31 

4.38 

4.43 

Instructional technology helps* teachers 
accommodate different learning styles. 

4.53 

4.63 

4.32 

4.46 

4.49 

Instructional technology helps* teachers 
to be more creative. 

4.50 

4.60 

4.37 

4.38 

4.46 

Instructional technology improves 
student learning comprehension.** 

4.30 

4.52 

4.20 

4.26 

4.32 

Instructional Technology increases 
student motivation and enthusiasm for 
learning. 

4.51 

4.56 

4.41 

4.48 

4.49 

In my experience, administrators support 
and encourage teachers to use instruc- 
tional technology in the classroom.*** 

3.96 

4.04 

3.72 

3.89 

3.90 

Teachers are generally eager to use 
instructional technology in the 
classroom. § 

3.47 

3.39 

3.45 

3.21 

3.38 

In general, the instructional programs 
I’ve seen are of good quality. §§ 

3.68 

3.92 

3.78 

3.54 

3.73 

Average 

4.17 

4.28 

4.07 

4.08 



| The program was called the NASA “Why?” Files during these years. 

* The previous years’ surveys used the word “enables” instead of “helps.” 

** The previous years’ wording was “increases student learning and comprehension.” 

***The previous years’ wording was “Administrators support and encourage teachers to use instructional 
technology in the classroom.” 

§ The previous years’ wording was “Teachers are generally positive about introducing/using instructional 
technology in the classroom.” 

§§ The previous years’ wording was “Most of these programs are of good quality.” 
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However, this promise is also tempered by a reality in which teachers are comparatively reluctant to 
use instructional technology in the classroom (average mean = 3.47) because of (1) lack of support from 
administrators (average mean = 3.90), (2) lack of good quality instructional technology programs (aver- 
age mean = 3.73), and from the current survey (3) the belief that the technology training provided has not 
improved their computer skills to a high degree (mean = 3.54; see table 1). Also consider that when asked 
to select from a list of barriers, those that inhabit or restrict the use of instructional technology in the 
classroom, respondents identified ( 1 ) lack of computers and computer access and (2) lack of time as the 
greatest offenders. Lastly, from the qualitative data come the following two recurring comments: (1) lack 
of available classroom time and (2) lack of control over what (instructional programs and materials) can 
and cannot be used in the classroom. In general, the power and position of these factors combine to lessen 
the use of instructional technology in the classroom and, in particular, the potential for educators to use 
the NASA SCI Files™. 

Topic 2. NASA Science Files™ Television/Video Programs 

The qualitative data indicate that all the programs are widely used (nearly 100 percent of the respon- 
dents) and that the programs are most often used to reinforce or to introduce a curriculum topic, objective, 
or skill (93 and 81 percent, respectively), and that we are on target with our grade level (used mostly in 
the 3 rd -6 th grades). 

NASA SCI Files™ is designed to enhance and enrich the “teaching and learning” of mathematics, sci- 
ence, and technology in grades 3-5. To do so, the NASA SCI Files™ (1) uses PBL to introduce students 
to scientific inquiry and the scientific method, (2) provides students the opportunity to simultaneously 
learn subject-matter and develop problem-solving skills while engaging in “real world” problems, (3) 
demonstrates workplace mathematics, science, and technology as a collaborative process, (4) raises stu- 
dent awareness of careers, and (5) overcomes students’ stereotyped beliefs by presenting women and 
minorities in challenging careers. The adoption of these goals is supported by the findings of the Educa- 
tional Research Service regarding Improving Student Achievement in Science. According to these find- 
ings, “Using real-life situations in science instruction through the use of technology (e.g., films, video- 
tapes, CDs, and DVDs) or through actual observation increases student interest in science, problem- 
solving skills, and achievement” (Cawelti, 1999). Are the established objectives for the NASA SCI 
Files™ being met? 

From figure 6 we see that 86 percent of respondents rated the overall quality of the NASA SCI Files™ 
programs “better than average,” 14 percent rated the programs “about average,” and no one rated them 
“worse than average.” Similar ratings were presented in figure 7 for the quality of the videos (79 percent 
“better than average”, 21 percent “about average”, and less that 1 percent “worse than average”). These 
high ratings have been consistent over the past 3 years. The responses to 13 questions that have been used 
for 4 years concerning the overall quality of the NASA SCI Files™ programs are presented in table 7. 
The overall average of the average means for the 13 questions is 4.46 (bottom right column), and the 
averages of the means for individual questions range from 4.29 to 4.60. Eleven of the 13 means for the 
2003-2004 broadcast season are higher than for the previous year, which indicates the program quality is 
moving in the right direction. An objective reading and interpretation of these data indicate that the 
objectives established for the NASA SCI Files™ are being met. 
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Table 7. Quality of NASA SCI Files™ Television/Video Programs Comparison Means 


Question 

2000-200 If 

2001-2002f 

2002-2003 

2003-2004 

Average 

The programs were well organized. 

4.65 

4.54 

4.36 

4.43 

4.50 

The programs were of good technical 
quality. 

4.68 

4.65 

4.44 

4.53 

4.58 

The programs made learning science and 
math interesting.* 

4.69 

4.61 

4.50 

4.59 

4.60 

The programs increased my students’ 
knowledge of science and math.** 

4.53 

4.59 

4.38 

4.39 

4.47 

The programs increased student motiva- 
tion and enthusiasm for learning.*** 

4.35 

4.35 

4.38 

4.50 

4.40 

The programs were a valuable instruc- 
tional aid. 

4.44 

4.57 

4.39 

4.57 

4.49 

The programs were appropriate for the 
specified grade level. § 

4.13 

4.50 

4.23 

4.37 

4.31 

The programs were easily incorporated 
into the curriculum. 

4.26 

4.20 

4.26 

4.44 

4.29 

The programs enhanced the integration 
of mathematics, science, and 
technology. §§ 

4.50 

4.69 

4.43 

4.48 

4.53 

The programs raised student awareness 
of careers that require mathematics, 
science, and technology. 

4.47 

4.44 

4.43 

4.30 

4.41 

The programs demonstrated the applica- 
tion of mathematics, science, and 
technology on the job. 

4.60 

4.52 

4.38 

4.35 

4.46 

The programs were aligned with national 
mathematics, science, and technology 
standards.§§§ 

4.64 

4.71 

4.47 

4.50 

4.58 

The programs presented females and 
minorities performing challenging 
engineering and scientific tasks.§§§§ 

4.34 

4.45 

4.26 

4.43 

4.37 

Average 

4.48 

4.52 

4.38 

4.45 

4.46 


t The program was called the NASA “Why?” Files during these years. 

* The previous years’ wording was “The programs made Teaming science’ interesting.” 

** The previous years’ wording was “The programs increased your students’ knowledge of science and math.” 
*** The previous years’ wording was “The programs increased student enthusiasm for learning.” 

§ The previous years’ wording was “The programs were developmentally appropriate for the grade level.” 

§§ The previous years’ wording was “The programs enhanced the integration of mathematics, science, and 
technology in the classroom.” 

§§§ The previous years’ wording was “The program content was aligned with national mathematics, science, and 
technology standards.” 

§§§§The previous years’ wording was “The programs presented women and minorities perfomiing challenging 
engineering and scientific tasks.” 
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As in past years, most (76 percent) of those surveyed indicated that the program length is “just right,” 
while 23 percent indicated that the programs are too long. Less than 1 percent indicated the programs are 
too short. This year 39 percent of users experienced difficulty obtaining the programs, a finding that 
represents a slight decrease over previous years. This response may reflect a lack of technology and 
knowledge of its use. 

Topic 3. NASA Science Files™ Educator Guides 

Of the respondents surveyed, 85 percent reported using the educator guides. When compared to other 
educator guides, 86 percent of respondents who reviewed the guides reported that the NASA SCI Files™ 
educator guides were better than average. (These data are comparable to the previous year’s data.) The 
comparison data for six questions rating the quality of the educator guides are presented in table 8. The 
overall average of the average means for the six questions is 4.47, and the averages of the means for indi- 
vidual questions range from 4.35 to 4.53. Five of the six means for the 2003-2004 broadcast season are 
higher than for the previous year, which indicates the program quality is moving in the right direction. 
Over the four years, respondents have reported that the educator guides were a valuable instructional aid 
(average mean of 4.53), that there was a good correlation between the educator guides and the video pro- 
grams (average mean of 4.52), and that the layout of the guides presented information clearly (average 
mean of 4.50). The lowest scoring question pertained to the ease of downloading the educator guides 
(average mean of 4.35). 

Table 8. Quality of NASA SCI Files™ Educator* Comparison Means 


Question 

2000-200 If 

2001-2002f 

2002-2003 

2003-2004 

Average 

The lesson guides* were a valuable 
instructional aid. 

4.57 

4.63 

4.48 

4.43 

4.53 

The lesson guides* were easy to 
download from the Internet. 

4.51 

4.21 

4.32 

4.36 

4.35 

The lesson guides correlated well with 
the video.** 

4.59 

4.57 

4.36 

4.55 

4.52 

The directions/instructions in the lesson 
guides were easy to understand.*** 

4.50 

4.48 

4.34 

4.48 

4.45 

The layout of the lesson guides* 
presented the information clearly. 

4.54 

4.56 

4.38 

4.52 

4.50 

The print and electronic resources in the 
lesson guides were valuable to me. § 

4.50 

4.46 

4.45 

4.54 

4.49 

Average 

4.54 

4.49 

4.39 

4.48 

4.47 


| The program was called the NASA “Why?” Files during these years. 

* In previous years, the lesson guides were called educator guides in all the questions. 

** The previous years’ wording was “The educator guides correlated with the video.” 

***The previous years’ wording was “The directions/instructions in the educator guides were easily understood.” 
§ The previous years’ wording was “The print and electronic resources in the educator guides were a valuable 
instructional aid.” 
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Comments taken from the qualitative data (appendix C) regarding the educator guides were largely 
positive; however, several respondents did indicate difficulty securing copies of the educator guides from 
the Internet, which is largely a function of the technology available in the school systems. An objective 
reading and interpretation of these data indicate that the educator guides for the NASA SCI Files™ are of 
high quality. 

Topic 4. NASA Science Files™ Problem-Based Learning Activities 

“PBL is a methodology based on the principle of using problems as the starting point for acquiring 
new knowledge. Pivotal to its effectiveness is the use of problems that create new learning experiences 
that reinforce and expand existing knowledge” (Lambros, 2002). The NASA SCI Files™ uses PBL to 
introduce students to science as inquiry and to the scientific method. Each NASA SCI Files™ program 
requires students to define the problem, perform research and investigations, formulate and test a 
hypothesis, perform experiments, collect and analyze data, draw conclusions, and find solutions to the 
problem. 


Table 9. Quality of NASA SCI Files™ PBL Activities Comparison Means 


Question 

2000-2001f 

2001-2002f 

2002-2003 

2003-2004 

Average 

The PBL activities were easily integrated 
into the curriculum.* 

4.22 

4.27 

4.26 

4.32 

4.27 

The PBL activities raised student aware- 
ness of careers that require mathematical, 
technological, and scientific knowledge.** 

4.34 

4.37 

4.29 

4.30 

4.33 

Students were able to complete the PBL 
activities in a reasonable amount of time. 

4.04 

4.03 

4.10 

4.06 

4.06 

The PBL activities accommodated various 
learning styles. 

4.22 

4.16 

4.27 

4.31 

4.24 

The content of the PBL activities was 
appropriate for my students. 

4.32 

4.24 

4.26 

4.22 

4.26 

The PBL activities enhanced the 
integration of mathematics, science, and 
technology.*** 

4.38 

4.44 

4.29 

4.48 

4.24 

Average 

4.25 

4.25 

4.25 

4.28 

4.23 


| The program was called the NASA “Why?” Files during these years. 

* The previous years’ wording was “The content of the PBL activities was easily integrated into the curriculum.” 
** The previous years’ wording was “The PBL activities raised student awareness of careers that require 
mathematical, scientific, and technological knowledge.” 

***The previous year’s wording was “The content of the PBL activities enhanced the integration of mathematics, 
science, and technology.” 


Overall, the NASA SCI Files™ PBL activities received high ratings for both their quality and content. 
The responses to six questions that have been used for four years concerning the overall quality of the 
PBL activities created for the NASA SCI Files™ programs are presented in table 9. The overall average 
of the average means for the six questions is 4.23, and the averages of the means for individual questions 
range from 4.06 to 4.33. Four of the six means for the 2003-2004 broadcast season are higher than for the 
previous year, which indicates the PBL activity quality is moving in the right direction. Over the four 


25 



years, respondents who used the PBL activities indicated that they raised student awareness of careers that 
require mathematical, technological, and scientific knowledge (average mean of 4.33), were easily inte- 
grated into the curriculum (average mean of 4.27), and that the content of the PBL activities was appro- 
priate for their students (average mean of 4.26). From figure 12 we see that 71 percent of respondents 
rated the overall quality of the NASA SCI Files™ PBL activities compared to others “better than average” 
and 29 percent rated the PBL activities “about average.” An objective reading and interpretation of these 
data indicate that the PBL activities for the NASA SCI Files™ are of high quality. 

Only 19 percent of the respondents indicated that they had used the PBL activities (comparable to pre- 
vious years). Although not specifically surveyed, we surmise that (1) lack of computers and computer 
access, (2) lack of available classroom time, and (3) lack of control over what (instructional programs and 
materials) can and cannot be used in the classroom are likely explanations (causes) for not using PBL 
activities. 

Topic 5. NASA Science Files™ Web Site 

Of the 203 respondents who replied when asked if they had viewed the NASA SCI Files™ web site, 
100 percent indicated that indeed they had. The responses to three questions that have been used for four 
years concerning the overall quality of the NASA SCI Files™ web site are presented in table 10. The 
overall average of the average means for the three questions is 4.41, and the averages of the means for 
individual questions range from 4.32 to 4.51. All three means for the 2003-2004 broadcast season are 
higher than for the previous year, which indicates the program quality is moving in the right direction. 
The highest average rating over the years indicates that the web site is visually appealing (average mean 
of 4.51). 


Table 10. Quality of NASA SCI Files™ Web Site Comparison Means 


Question 

2000-2001f 

2001-2002f 

2002-2003 

2003-2004 

Average 

The NASA SCI Files™ web site is 
visually appealing.* 

4.67 

4.52 

4.36 

4.48 

4.51 

The NASA SCI Files™ web site is easy to 
navigate.** 

4.49 

4.34 

4.20 

4.23 

4.32 

The links to other sites/pages are current/ 
up to date.*** 

4.47 

4.60 

4.24 

4.33 

4.41 

Average 

4.54 

4.49 

4.27 

4.35 

4.41 


t The program was called the NASA “Why?” Files during these years. 

* In previous years, this question was worded as “The web site is visually appealing.” 

** In previous years, this question was worded as “The web site is easily navigated.” 

***In previous years, this question was worded as “The links to other sites/pages are current.” 


An objective reading and interpretation of these data indicate that the web site for the NASA SCI 
Files™ is of high quality. 

Topic 6. Overall Assessment of the NASA Science Files™ 

The overall assessment of the NASA SCIence Files™ series was very positive. Of the respondents, 
98 percent indicated that they would recommend NASA SCIence Files to a friend. Ninety-six percent 
of respondents reported that the programs were successful in educating and informing others about what 
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NASA does. When asked about the credibility of the information contained in the programs, 95 percent of 
respondents indicated that they found the information veiy credible. 

Topic 7. Videoconferencing 

The 2003-2004 evaluation of the NASA SCIence Files™ marks the first year we inquired about video- 
conferencing. The data collected indicate that educators do have access to video-conferencing equipment 
and would be interested in a NASA-sponsored videoconference. When asked if they had access to video- 
conferencing equipment for instruction, 36 of 37 respondents indicated that they did. Of these respon- 
dents, 34 indicated that they would be interested in having their students participate in a NASA-sponsored 
videoconference. When asked about a language preference for such a videoconference, 89 percent of 
respondents specified English as their preferred language, 3 percent selected Spanish, and 8 percent indi- 
cated a preference for both English and Spanish. 

Concluding Remarks 

In marketing, there are three significant measures of success: (1) repeat purchasing, (2) whether or not 
a product is recommended to others, and (3) how that product fares in a competitive environment. The 
fact that 30 participants in this year’s survey had used the programs for 2 years, 21 for 3 years, 12 for 
4 years, and 6 for 5 years supports the first measure of success: repeat purchasing. That 98 percent of the 
respondents had or would recommend the NASA SCIence Files™ to a colleague supports the second 
measure: product recommendation. The third measure, “How the NASA SCIence Files fares in a com- 
petitive environment?” is demonstrated by the 86 percent of respondents’ reporting that the NASA 
SCIence Files was better than existing and similar (science) instructional programming. Collectively, 
the findings of this report support the continued production of the NASA SCIence Files™. 

Based on the quantitative and qualitative data, the following seven recommendations are offered as 
part of the ongoing effort to continuously improve the NASA SCIence Files . 

Data 1 : Although there is general agreement among the respondents that instructional technology helps 
educators teach more effectively and enriches the learning process, respondents indicated that certain 
factors — (1) lack of computers and computer access, (2) lack of available classroom time, and (3) lack of 
control over what (instructional programs and materials) can be used in the classroom — combine to limit 
the use of instructional technology programs such as the NASA SCI Files™. These data appear to 
correlate with data obtained from several large-scale (national) instructional technology studies and 
indicate that the views held by respondents to this study regarding instructional technology are very 
similar to those held by their peers. What is not known is that if these three factors were removed or 
otherwise mitigated, would the use of instructional technology among the respondents increase? That 
said, it might be useful to add questions to the existing survey to determine the extent to which 
respondents have been trained to use or are otherwise predisposed to use instructional technology in the 
classroom. 

Recommendation 1: Using an appropriate methodology, determine (1) the extent to which registered 
users of the NASA SCI Files have been trained to use and to integrate instructional technology into the 
curriculum and (2) whether a professional development (i.e., training) component should be developed for 
the NASA SCI Files™. 
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Data 2: The NASA SCI Files™ is a series of 60-minute programs (for students in grades 3-5) that are 
research-, inquiry-, standards-, teacher-, and technology-based. The programs are designed to introduce 
students to NASA; integrate mathematics, science, and technology through the use of PBL, scientific 
inquiry, and the scientific method; and to motivate students to become critical thinkers and active problem 
solvers. Overall, survey respondents agree (1) that the programs in the 2003-2004 series met their stated 
objectives; (2) that the length of the programs (60 minutes) was neither too long nor too short; and (3) that 
the programs are used most often to reinforce a topic, objectives, or skill. Survey respondents reported 
that the educator guides correlated well with the instructional broadcast, that the guides were a valuable 
aid, and that they were easy to download from the Internet. Survey participants also gave the PBL 
activities and the NASA SCI Files™ web site high marks. 

Survey participants consider the NASA Science Files™ a beneficial (instructional) resource that 
enhances and enriches teaching and learning. Collectively, the data support the continued production of 
the series. It is important to note that the NASA SCI Files ranks well above average with regard to 
national trends in instructional technology and programming and is viewed as a valued resource by its 
users. 

Recommendation 2: As part of conference attendance and especially as part of any conference 
presentation, it might be instructive to conduct interviews with educators as a way of ( 1 ) learning more 
about the suitability/usability of the NASA Science Files™ and of (2) identifying barriers that might 
prohibit or inhibit its use, such as “a fixed curriculum” or “the amount of time available to teach science.” 
Lastly, it seems that increased use of the programs might result from greater explanation and 
demonstration of the NASA Science Files . Therefore, participation in pre-service and in-service 
educational workshops and as part of technology exhibits might result in increased use. 

Data 3: Several respondents indicated they had difficulty acquiring/obtaining the programs: “Our PBS 
station does not air the NASA programs” was most frequently reported. Several homeschoolers reported 
having problems acquiring the programs. One respondent complained (bitterly) about the difficulty 
encountered obtaining the NASA SCI Files™ programs from a NASA ERC. 

Recommendation 3: Conduct an investigation for the purpose of ascertaining the nature of the 
problems/difficulties registered users have receiving/obtaining the NASA SCI Files programs and 
determine what can be done to resolve them. 

Data 4: The lowest scoring question pertained to “the ease of downloading the educator guides.” 
Comments taken from the qualitative data regarding the educator guides were largely positive; however, 
several respondents did indicate difficulty securing copies of the NASA SCI Files™ educator guides from 
the Internet. 

Recommendation 4 : Conduct an investigation designed to determine whether (1) the difficulties users 
experience securing copies of the educator guides are attributable to the “host” site, and (2) in general, 
what actions/recommendations can be taken/offered to help users more “easily” secure copies of the 
NASA SCI Files™ educator guides. 

Data 5: Although not specifically stated, we infer from the data that (1) lack of computers and computer 
access, (2) lack of available classroom time, and (3) lack of control over what (instructional programs and 
materials) can and cannot be used in the classroom are likely explanations (causes) for not using the 
NASA SCI Files™ PBL activities. 
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Recommendation 5: Determine (1) the factors responsible for “non-use” of the PBL activities by 
registered users and (2) what can be done to increase usage of the NASA SCI Files PBL activities. 

Data 6: The survey indicated that the NASA SCI Files™ web site could be improved by making items 
faster to download. Two factors are relevant to the discussion: (1) concerning the Internet, download 
speed is related to and controlled by the connection speed of the user’s service provider and system, and 
(2) the process is controlled, to a limited extent, by the web (host) site. 

Recommendation 6: Determine (1) the factors responsible for “non-use” of the NASA SCI Files™ web 
site by registered users and (2) what can be done to increase web site usage. 

Data 7: Of these respondents, 34 indicated that they would be interested in having their students 
participate in a NASA-sponsored videoconference. When asked about a language preference for such a 
videoconference, 88 percent of respondents specified English as their preferred language, 2 percent 
selected Spanish, and 8 percent indicated a preference for both English and Spanish. 

Recommendation 7: Attempts should be made to follow up with the respondents who indicated an 
interest in having their students participate in a NASA-sponsored videoconference. 

Data 8: As of September 30, 2004, 145,600 (formal and informal) educators, representing 4.3 million 
students and 384 television stations, with a combined (potential) audience of 157.6 million, were 
registered users of the NASA SCI Files™. 

Recommendation 8: Starting with the 2004-2005 NASA SCI Files™ season, the minimum number of 
completed surveys should be set at 300 (with 350 being ideal). Continue efforts to increase the number of 
registered informal educators and the number of television stations airing the NASA SCI Files , with 
special emphasis placed on increasing the number of PBS stations airing the NASA SCI Files . 
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Appendix A 


National Aeronautics and 
Space Administration 

Langley Research Center 

Hampton, VA 23681-2199 


Educational Product 
Educators Grades 3-5 

EP-2003-07-1 0-LARC 



Free Classroom 
Mentors 

The SWE provides 
classroom mentors to 
assist with the hands- 
on activities and 
scientific concepts. For 
more information, visit 
the NASA SCI Files™ 
web site or e-mail 
kimlien.vu@swe.org. 


Season 2003-2004 m 

The NASA Science Files™ is an annual series of FREE 
Emmy®- award-winning instructional programs consisting 
of broadcast, print, and online elements. Emphasizing 
research- and standards-based instruction, Problem-Based 
Learning, and scientific inquiry, the series seeks to motivate 
students in grades 3-5 to become critical thinkers and 
active problem solvers. Each program supports the 
national mathematics, science, and technology standards 
and has three components that include (1 ) a 60-minute 
television broadcast, which can be viewed live or taped for 
later use; (2) a companion educator guide; and (3) an 
interactive web site featuring a Problem-Based Learning 
activity that enables students to further explore topics 
presented in the broadcast. The web site also contains a 
wealth of instructional resources. 

Use the Power of Technology to 
Make the NASA Connection* 

This opportunity is offered as a part of NASA LIVE™: (Learning through interactive 
Videoconferencing Experiences), a program produced by NASA Langley's Center 
for Distance Learning. 

Now, in addition to the broadcast, educator guide, and web 
activity, your students can participate in a FREE virtual field trip. 
This 45-minute, interactive videoconference connects them to the 
NASA experts featured in each new NASA SCI Files™ program, the 
latest research, exciting demonstrations, and more. See reverse 
side for a list of scheduled events. Don't 
miss this exciting opportunity to bring 
math, science, and technology to life!! 


* Students must have viewed the NASA SCI Files™ 
broadcast prior to participating. Participation is 
limited to the first four schools for each original 
event. Additional sessions can be scheduled upon 
your request.To register for an event and additional 
information, visit http://live.larc.nasa.gov. 


Check the back for a list of programs in the 
2003-2004 NASA SCI Files™ series and for more 
information about obtaining the programs. 


www.pbs.org 
www.sbo.hampton.kl 2.va.us 


1 Register 

Register online at 

scifiles.larc.nasa.gov 

2 Access 

How can I get the 
>> television broadcast? 


The NASA SCI Files™ 
programs air on PBS, NASA 
TV, and many Cable Access 
Channels. Check our web 
site for stations in your area. 

The programs are up-linked 
in KU- and C-band.The 
satellite coordinates are 
listed on the NASA SCI 
Files™ web site. 

Programs are available 
online via video streams. 
See web site for details. 

Video copies of the 
broadcast can be obtained 
from the NASA Educator 
Resource Center in your 
state, http://education. 
nasa.gov/ercn, or from 
NASA CORE, 

http://core.nasa.gov or 

call toll free, 

1 -866-776-CORE. 


\ 3 Integrate 

Integrate NASA SCI Files™ 
§■ into your classroom to 
~ inspire your students and 

z to enhance learning. 

3S 

D 

TJ 
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I The Case of the 

Challenging Flight (R)* 

Starts airing: Wed., September 1 7, 2003 
1 1:00 a.m.-12 Noon EDT 
The tree house detectives accept a challenge 
to compete in an "Egg-tra-ordinary" airplane 
contest. The detectives design and build an 
airplane by using common household materi- 
als as they learn about the four basic compo- 
nents of flight: lift, thrust, drag, and weight. 
Mathematics Standards: Measurement; Data 
Analysis and Probability; Problem Solving; 
Communication 

Science Standards: Science and Inquiry; 
Physical Science; Science and Technology; 
Science in Personal and Social Perspectives 
Technology Standards: Nature of 
Technology; Technology and Society; Design; 
Abilities for a Technological World;The 
Designed World 

2 The Case of the Wacky 
Water Cycle 

Starts airing: Wed., October 1 5, 2003 
1 1 :00 a.m.-l 2 Noon EDT 
NASA LIVE event: November 1 2, 2003 
The tree house detectives' efforts to raise 
money with a car wash dry up when a 
summer drought hits the city. With the help of 
Problem-Based Learning (PBL), the entire 
group learns all about the water cycle, the 
water table, global climates and much more to 
get their project flowing again. 

Mathematics Standards: Numbers and 
Operations; Measurement; Data Analysis and 
Probability; Problem Solving; Communication 
Science Standards: Science and Inquiry; 

Earth and Space Science; Science and 
Technology; Science in Personal and Social 
Perspectives 

Technology Standards: Nature of 
Technology; Technology and Society; Design; 
Abilities for a Technological World;The I 

Designed World 1 

3 The Case of the "Wright" 
Invention (R)* 

Starts airing: Wed., November 19,2003 
1 1 :00 a.m.-l 2 Noon EST 
Travel back in time with the tree house detec- 
tives to learn about the process of invention 
from two of the greatest inventors of all time, 
Orville and Wilbur Wright. The tree house 
detectives find that inventing is not as easy as 
it seems, and it really does take the "Wright" 
stuff to be a good inventor. 

Mathematics Standards: Numbers and 
Operations; Measurement; Data Analysis and 
Probability; Problem Solving; Communication; 
Representation 

Science Standards: Science and Inquiry; 
Physical Science; Science and Technology; 
Science in Personal and Social Perspectives; 
History and Nature of Science 
Technology Standards: Nature of 
Technology; Technology and Society; Design; 
Abilities for a Technological World;The 
Designed World 


4 The Case of the 
Disappearing Dirt 

Starts airing: Wed., December 10,2003 
11:00 a.m.-l 2 Noon EST 
NASA LIVE event: January 21,2004 
Summer fun suffers a setback when the tree 
house detectives discover that their favorite 
spot on the beach is shrinking. It is "match, 
set, point" as the detectives dig in and learn all 
about erosion, rocks, and natural preservation. 
Mathematics Standards: Numbers and 
Operations; Measurement; Data Analysis and 
Probability; Problem Solving; Communication 
Science Standards: Science and Inquiry; 
Physical Science; Earth and Space Science; 
Science and Technology 
Technology Standards: Nature of 
Technology; Technology and Society; Abilities 
for a Technological World; The Designed World 

5 The Case of the Galactic 
Vacation (R)* 

Starts airing: Wed., January 21,2004 
11:00 a.m.-l 2 Noon EST 
The tree house detectives go galactic with 
their latest project. Learn how long it will take 
to travel to Mars and how the Moon affects 
the Earth. Come visit the largest radio tele- 
scope in the world and help look for intelli- 
gent life in the universe. Join the tree house 
detectives for an "out-of-this-world" vacation 
as they explore the future of space travel. 
Mathematics Standards: Geometry; 
Measurement; Data Analysis and Probability; 
Problem Solving 

Science Standards: Science and Inquiry; Life 
Science; Earth and Space Science; Science and 
Technology 

Technology Standards: Nature of 
Technology; Technology and Society; Design; 
The Designed World 

The Case of the Prize- 
Winning Plants 

Starts airing: Wed., February 1 8, 2004 
11:00 a.m.-l 2 Noon EST 
NASA LIVE event: March 17,2004 
Everyone's green thumb is put to the test as 
the tree house detectives attempt to grow 
award-winning plants for the upcoming fair. 
The tree house becomes a greenhouse as the 
detectives experiment with soil, plant and ani- 
mal life cycles, and genetics to grow the per- 
fect plant. 

Mathematics Standards: Numbers and 
Operations; Measurement; Data Analysis and 
Probability; Problem Solving; Communication 
Science Standards: Science and Inquiry; 
Physical Science; Earth and Space Science; Life 
Science; Science and Technology 
Technology Standards: Nature of 
Technology;Technology and Society; Abilities 
for a Technological World; The Designed World 


7 The Case of the Powerful 
Pulleys (R)* 

Starts airing: Wed., March 1 7, 2004 
11:00 a.m.-l 2 Noon EST 
One of the tree house detectives has had an 
accident and cannot get into the tree house. 
Using Problem-Based Learning, the rest of the 
gang investigates the world of simple 
machines and physical science and "pulls" 
together to get everyone into the tree house. 
Mathematics Standards: Numbers and 
Operations; Measurement; Algebra; Data 
Analysis and Probability; Problem Solving; 
Connections 

Science Standards: Science and Inquiry; 
Physical Science; Science and Technology 
Technology Standards: Nature of 
Technology; Technology and Society; Design; 
Abilities for a Technological World; The 
Designed World 

8 The Case of the Radical Ride 

Starts airing: Wed., April 1 4, 2004 
11:00 a.m.-l 2 Noon EDT 
NASA LIVE event: May 1 9, 2004 
The tree house detectives' latest project about 
alternative forms of transportation takes on 
new relevance when they get stuck in traffic. 
Join the crew as they learn about energy, com- 
posite materials, and technology in their quest 
for hassle-free traveling. 

Mathematics Standards: Numbers and 
Operations; Measurement; Data Analysis and 
Probability; Problem Solving; Communication 
Science Standards: Science and Inquiry; 
Physical Science; Earth and Space Science; 
Science and Technology 
Technology Standards: Nature of 
Technology; Technology and Society; Design; 
Abilities for a Technological World; The 
Designed World 

9 The Case of the Phenomenal 
Weather (R)* 

Starts airing: Wed., May 1 9, 2004 
11:00 a.m.-l 2 Noon EDT 
Join the tree house detectives as they plan a 
trip to the Caribbean and encounter problems 
trying to predict the weather. In this case the 
tree house detectives will learn about violent 
storms, such as hurricanes and tornadoes, 
weather fronts, global wind patterns, and cli- 
mates. While solving the case, they will discov- 
er that predicting the weather is not pre- 
dictable at all! 

Mathematics Standards: Numbers and 
Operations; Algebra; Geometry; Measurement; 
Data Analysis and Probability; Problem 
Solving; Representation 
Science Standards: Science and Inquiry; 
Physical Science; Life Science; Earth and Space 
Science; Science and Technology 
Technology Standards: Nature of 
Technology; Technology and Society; Abilities 
for a Technological World; The Designed World 


(R) Indicates a repeat program from the 2002-2003 season. 2003-2004 Series • Grades 3 - 5 • http://scifiles.larc.nasa.gov [cc] 
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Appendix B 

2003-2004 NASA SCIence Files™ Evaluation 


Please confirm the following Information: 

First Name 


Last Name 


Address: 


City: 


State: 


Zip: 


Email: 



Instructional Technology fin Generali and Teaching 

Please indicate the extent to which you Disagree or Agree with the following 
statements about using instructional technology in the classroom. 

1. In general, instructional technology helps teachers to teach more effectively. 
(Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree U L LI L L Strongly Agree Lj 

2 . Instructional technology helps teachers accommodate different learning styles. 
(Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

3 . Instructional technology helps teachers to be more creative. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

4 . Instructional technology improves student learning comprehension. (Check 
one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

5 . Instructional technology increases student motivation and enthusiasm for 
learning. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 


6 . In my experience, administrators support and encourage teachers to use 
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instructional technology in the classroom. (Check one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

7 . Teachers are generally eager to use instructional technology in the classroom. 
(Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

8 . In general, the instructional programs I've seen are of good quality. (Check 
one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

9 . The technology training provided by my school division has improved my 
computer skills. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

10 . Are there any barriers that keep you from using more technology in your 
teaching? (Check all that apply.) 


No barriers 

Not enough/limited access to computers 
Not enough computer software 

Lack of time available in school for technology projects 
Lack of technical support/help for technology projects 

NASA Science Files™ Programs 

These questions refer to the 9 programs in the 2003-2004 NASA Science Files™ 
series. 

11 . Have you used any of the 9 NASA Science Files™ 2003-2004 programs? 
(Check one.) 



No (Skip to Q48.) 

n 

No, but I may in the future (Skip to Q48.) 

12 . What grade level(s) viewed the NASA science Files™ programs? (Mark all 
that apply.) 
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K 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 


13 . Compared to other instructional programming, is the OVERALL quality of 
NASA Science Files™: (Check one.) 


□ 

□ 

□ 


Better than average 


About average 


Worse than average 


14 . Compared to the VIDEO in other instructional programs, were the NASA 
Science Files™ VIDEOS: (Check one.) 


□ 


□ 


Better than average 
About average 


Worse than average 

15 . Did your students view Program 1 - The Case of the Challenging Flight? 
(Check one.) 


□ 

□ 


Yes 

No 


16 . Did your students view Program 2 - The Case of the Wacky Water Cycle? 
(Check one.) 


□ 


Yes 


17 . Did your students view Program 3 - The Case of the "Wright" Invention? 
(Check one.) 


□ 

□ 


Yes 

No 


18 . Did your students view Program 4 - The Case of the Disappearing Dirt? 
(Check one.) 


L3 


Yes 


19 . Did your students view Program 5 - The Case of the Galactic Vacation? 
(Check one.) 
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l: 

□ 


Yes 

No 


20 . Did your students view Program 6 - The Case of the Prize-Winning Plants? 
(Check one.) 


□ 


Yes 


□ 


No 


21 . Did your students view Program 7 - The Case of the Powerful Pulleys? (Check 
one.) 


□ 


Yes 

No 


22 . Did your students view Program 8 - The Case of the Radical Ride? (Check 
one.) 


□ 


Yes 




No 


23 . Did your students view Program 9 - The Case of the Phenomenal Weather? 
(Check one.) 


□ 

□ 


24 . In general, how did you use the NASA Science Files™ programs? Were they 
ever used to introduce a curriculum topic, objective, or skill? (Check one.) 


□ 


Yes 


□ 


No 


25 . Were the programs used to reinforce a curriculum topic, objective, or skill? 
(Check one.) 


□ 

□ 


Yes 

No 


26 . Were they ever used as a special interest topic? (Check one.) 


□ 


Yes 
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□ 


No 


27 . Were they used as a break from classroom routine? (Check one.) 


□ 

LZ 


Yes 

No 


28 . How do you usually receive the NASA science Files™ programs? (Check 
one.) 


□ 

□ 

□ 

□ 

U 

□ 


From PBS/ITV 
By downloading it 

A media specialist taped it to show later. 

I (or another teacher) taped it to show later. 

I get the tapes from NASA's Educator Resource Center (ERC). 
Other. 


29 . How many years have you used NASA Science Files™ programs? 
(1 = 1 or less) 


years 


30 . Did you experience any difficulty obtaining any of the 2003-2004 NASA 
science Files™ programs? (Check one.) 


□ 

□ 


Yes 


No 


Please indicate the extent to which you disagree or agree with the following 
statements concerning the 2003-2004 NASA Science Files™ series programs. 

31 . The programs were well organized. (Check one.) 


Strongly Disagree L] Lj Lj Lj Lj Strongly Agree 

32 . The programs were of good artistic quality. (Check one.) 


No Opinion 
□ 


3 

CJ 


4 

□ 


Strongly Disagree [][_][][][_] Strongly Agree 

33 . The programs were of good technical quality. (Check one.) 


No Opinion 

□ 


Strongly Disagree 


Strongly Agree 


No Opinion 
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□ □ 

c 

□ 

□ 

□ 

34. The programs made learning science and math interesting. 

(Check one.) 

1 2 

3 

4 

5 


No Opinion 

Strongly Disagree Lj Lj 

□ 

□ 

□ 

Strongly Agree 

□ 

35. The programs helped me to accommodate different learning styles. (Check 

one.) 






1 2 

3 

4 

5 


No Opinion 

Strongly Disagree LJ LJ 

□ 

□ 

□ 

Strongly Agree 

□ 

36. The programs increased my students' knowledge of science and math. 

(Check one.) 






1 2 

3 

4 

5 


No Opinion 

Strongly Disagree E_j Lj 

□ 

□ 

□ 

Strongly Agree 

□ 

37. The programs increased student motivation and enthusiasm for learning. 

(Check one.) 






1 2 

3 

4 

5 


No Opinion 

Strongly Disagree [_] [J 

□ 

□ 

□ 

Strongly Agree 

□ 

38. The programs were a valuable instructional aid. (Check one.) 

1 2 

3 

4 

5 


No Opinion 

Strongly Disagree Lj Lj 

□ 

□ 

□ 

Strongly Agree 

□ 

39. The programs were appropriate for the specified grade level 

1. (Check one.) 

1 2 

3 

4 

5 


No Opinion 

Strongly Disagree [J 

c 

12 

□ 

Strongly Agree 

□ 

40. The programs were easily incorporated into the curriculum. 

(Check one.) 

1 2 

3 

4 

5 


No Opinion 

Strongly Disagree EJ 

□ 

12 

□ 

Strongly Agree 

□ 

41. The programs enhanced the integration of mathematics, science, and 

technology. (Check one.) 






1 2 

3 

4 

5 


No Opinion 

Strongly Disagree 

□ 

12 

l; 

Strongly Agree 

12 

42. The programs raised student awareness of careers that require mathematics, 

science, and technology. (Check one.) 






Strongly Disagree 


2 3 4 


Strongly Agree 


No Opinion 
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□ l: c ej l: 


□ 


43 . The programs demonstrated the application of mathematics, science, and 
technology on the job. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree 

44 . The programs were aligned with national mathematics, science, and 
technology standards. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree LJ Lj L] LJ LJ Strongly Agree [_] 

45 . The programs presented females and minorities performing challenging 
engineering and scientific tasks. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree U Lj Lj Lj L Strongly Agree LJ 

46 . Is the 60-minute length of the programs: (Check one.) 


L3 

□ 

□ 


Too short 


Just right 


Too long 


47 . Please add any other comments you have concerning the 2003-2004 NASA 
Science Files™ programs. 




About the Lesson Guides 


48 . Did you use the lesson guides for any of the 2003-2004 NASA science Files™ 
programs? (Check one.) 


□ 


Yes 


No (Skip to Q57.) 

r 

No, but I may in the future (Skip to Q57.) 

49 . Compared to other LESSON GUIDES, were NASA Science Files™ lesson 
guides: (Check one.) 


□ 


Better than average 
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□ 

□ 

□ 


About average 
Worse than average 
I didn't review the guides 


Please indicate the extent to which you disagree or agree with the following 
statements about the LESSON GUIDES. 


50 . The lesson guides were a valuable instructional aid. (Check one.) 


1 2 3 4 5 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree 


No Opinion 

□ 


51 . The lesson guides were easy to download from the Internet. (Check one.) 


1 2 3 4 5 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree 

52 . The lesson guides correlated well with the video. (Check one.) 


No Opinion 

□ 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

53 . The directions/instructions in the lesson guides were easy to understand. 
(Check one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

54 . The layout of the lesson guides presented the information clearly. (Check 
one.) 


1 2 3 4 5 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree 


No Opinion 

□ 


55 . The print and electronic resources in the lesson guides were valuable to me. 
(Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

56 . Please add any other comments you have concerning the lesson guides. 


The On-Line Problem-Based Learning (PBL) Activities 

57 . Did your students use any of the PBL activities found on the NASA science 
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Files™ web site? (Check one.) 



No (Skip to Q67.) 

rj 

No, but I may in the future (Skip to Q67.) 

58. Compared to other PBL activities, was the quality of the PBL activities for 
NASA Science Files™: (Check one.) 


□ 


□ 


□ 


Better than average 
About average 
Worse than average 


I didn't review the web-based activities 

Please indicate the extent to which you disagree or agree with the following 
statements about the PBL activities. 

59. The PBL activities are a valuable instructional aid. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree [_] 


60. The PBL activities were easily integrated into the curriculum. (Check one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

61. The PBL activities raised student awareness of careers that require 

mathematical, technological, and scientific knowledge. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

62. Students were able to complete the PBL activities in a reasonable amount of 

time. (Check one.) 


1 2 3 4 5 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree 


No Opinion 
□ 


63. The PBL activities accommodated various learning styles. (Check one.) 


1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

64. The content for the PBL activities was appropriate for my students. (Check 
one.) 
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1 2 3 4 5 

Strongly Disagree Lj Lj Lj Lj Strongly Agree 


No Opinion 
□ 


65 . The PBL activities enhanced the integration of mathematics, science, and 
technology. (Check one.) 

1 2 3 4 5 No Opinion 

Strongly Disagree Lj Lj Lj Lj Lj Strongly Agree Lj 

66 . Please add any other comments you have concerning the PBL activities. 


The NASA Science Files™ Web Site 

67 . Have you looked at the NASA science Files™ web site? (Check one.) 

U V.. 

r ^ 

No (Skip to Q73.) 

These questions pertain to the web site for the 2003-2004 NASA Science Files™ 
series. Please indicate the extent to which you disagree or agree with the 
following statements. 

68 . The NASA Science Files™ web site is visually appealing. (Check one.) 


1 

Strongly Disagree Lj 

2 

□ 

3 

□ 

4 

□ 

5 

r Strongly Agree 

No Opinion 
□ 

69 . The NASA Science Files™ web site is easy to navigate. (Check one.) 

i 

2 

3 

4 

5 

No Opinion 

Strongly Disagree Lj 

□ 

□ 

□ 

r ] Strongly Agree 

□ 

70 . The links to other sites/pages are current/up-to-date. (Check one.) 

i 

2 

3 

4 

5 

No Opinion 

Strongly Disagree Lj 

□ 

□ 

□ 

Lj Strongly Agree 

□ 

71 . The external links are good resources for further exploration. (Check one.) 

l 

2 

3 

4 

5 

No Opinion 

Strongly Disagree Lj 

□ 

□ 

□ 

[ Strongly Agree 

□ 


72 . The web site complements the broadcast/video. (Check one.) 
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No Opinion 
□ 


1 2 3 4 5 

Strongly Disagree Lj LJ Lj Lj Lj Strongly Agree 


Overall Assessment 


73 . If a colleague inquired, would you recommend NASA Science Files™ to 
him/her? (Check one.) 


□ 

□ 


74 . One of our goals is to educate and inform others about what NASA does. Do 
you think NASA Science Files™ has been successful in this regard? (Check one.) 


□ 


□ 


Yes 

No 


75 . In your opinion, was the information about NASA contained in the programs: 
(Check one.) 


□ 

□ 

□ 

□ 


Very credible 
Somewhat credible 
Not credible 
I'm unable to judge 


Videoconferencing 

76 . Do you have access to videoconferencing equipment for your instruction? 
(Check one.) 


□ 


□ 


Yes 

No (Skip to Q79.) 


77 . In the future, would you be interested in having students participate in a 
NASA-sponsored video conference? (Check one.) 


□ 

□ 


78 . For a NASA-sponsored video conference, what would be your language 
preference? (Check one.) 


English 
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Spanish 

n 

Both English and Spanish 

Demographics 

These questions tell us whether respondents with different backgrounds or 
characteristics have different opinions about instructional technology and NASA 
science Files™. 

79 . Your gender? 


l; 

ej 


Female 

Male 


80 . Do you have a personal computer at home? (Check one.) 


81 . Are you a member of a national professional education organization (e.g., 
ASDC, NMSA, NCTM, NSTA)? (Check one.) 


V 

□ 


82 . How many years have you been teaching? (Enter number of years) 


years (1 = 1 or less) 


83 . Employment? (Please check only one.) 


□ 

□ 

□ 

□ 

□ 

□ 


Public school 

Private or Parochial school 
Bureau of Indian Affairs school 
College/ University 
Home School -► Please skip to 085. 

Informal Educator (e.g., museum, 4-H, etc.) — > Please skip to Q85. 


84 . Do you have any of the following titles? (Check one for each title.) 
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□ 


□ 



Yes 

No 

Classroom Teacher 

□ 

□ 

Yes 

No 

Pre-service Educator 

□ 

□ 

Yes 

No 

Math Coordinator 

u 

□ 

Yes 

No 

Science Coordinator 

□ 

□ 

Yes 

No 

Librarian/Media Specialist 

□ 

□ 

Yes 

No 

Technology Program Coordinator 

□ 

□ 

Yes 

No 

Distance Learning Coordinator 

EJ 

□ 

Yes 

No 

Curriculum Coordinator 


85 . School Location? (Please check only one.) 


□ 

□ 

□ 


Rural 

Suburban 

Urban 


86 . What is the highest degree you have earned? (Please check only one.) 


□ 


□ 


High School Diploma 
Associates (2-Year) Degree 


□ 


□ 


□ 


Baccalaureate (BA/BS) Degree 
Masters/Masters Eguivalency 
Doctorate 


87 . Your ethnicity? (Please check only one.) 


□ 

□ 

□ 

□ 

□ 


African American 
Asian 

Caucasian/ White 
Hispanic 
Native American 
Pacific Islander 


Other 


88 . Your age? (Check one.) 


Under age 20 


□ 

□ 


Twenties 
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Thirties 


□ 


□ 


□ 


Forties 

Fifties 

Sixties or older 


OMB approval number: (0MB 2700-0012) 


Appendix C 


The responses below were given as "Other” comments concerning the 2003-2004 NASA SCIence 
Files ™ programs . 


I think that they are wonderful, and I wish that’s all we did! 

This was the first year that I have used one of the case files and it was a GREAT success with the 
students. They all learned so much and had a great time. As an educator, it was a great aid for the 
lesson being taught. Please keep up the good work. 

I have used the NASA SCIence Files for the past 3 years and continue to be impressed by the quality of 
the learning activities that are contained within it. I have taught in the past a gifted science class at the 
elementary level and have used the SCI Files programs as a spring board from which we jump into 
different topics of interest. The activities are easily customizable and are very motivating to the 
students. 

Need to be broken into segments so they can be used in parts throughout the lesson 

Great spaced out with experiments in between viewing 

I wish I was still receiving a hard copy through the mail because it is difficult to download the files. 

Keep up the good work 

I would prefer that the programs be 2-30 minute presentations that could be viewed over two days 
without interruption 

Will they ever be available in DVD format? 

The background music is too “jumpy,” therefore distracting, and frequently overwhelms the spoken 
words, therefore interfering with learning. Back off on the loud frenetic music, and the kids will be 
able to focus better on the actual concept being presented. (They won’t have to “listen through.”) 

Actors talk too fast. 

I was not aware of the lesson guides for the SCI files programs. I would like to have used them. 

I have not gotten the new program video. I cannot comment on them since I do not have them. 
Hopefully NASA will have them when I go tomorrow. 

I had difficulty downloading the video. We received video but could not get the audio to work. I will 
try to order the videos I need for next year. 

The videos are great, but I’m not too happy with the fact they are no longer sent to teachers in VHS 
format. 

I’m not an instructor. We downlink the programs and air them on a taped delay basis on our college’s 
educational cable channel. 

The background sound is distracting. It is too much and too loud. The people talk too fast. My kids 
like to take notes, and when the speakers go too fast as they compete with the background music, the 
kids get frustrated. 

They are very easy to include into our curriculum. The students were very involved in the lessons. 

My students really enjoyed the programs and the online investigations. 

Taping the shows was the barrier. Could we purchase them pre-taped in the future? 

My 2nd graders are interested but limited in background for the most part, so I have to explain a great 
deal. 
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Excellent! My only problem is I’m changing schools in the fall and must leave my tapes behind! 

I conduct share fairs in which my sixth graders teach various math and science concepts and skills to 
the younger students. The NASA Science Files provide additional resources and credibility for my 
students. I use them for background information, additional inquiry-based techniques, and share them 
with colleagues, 

I really enjoyed the information. 

E-mail announcement of broadcast times often arrived too late to arrange for taping by media services. 

Would like to have the video tapes sent to me since we do not get the programs here on our television 
PBS station 

I reviewed Spanish video tapes made available at the CABE conference in San Jose, California in 
March or 2004. Watch for enunciation and rate of speech. Some important statements were lost to 
poor enunciation or spoken too fast to understand clearly. 

Can be adapted to suit needs easily. 

Keep up the good work. 

Excellent Quality! 

None 

They are wonderful. Please make more. 

I just wish I had more time to implement the program. 

I would like to start receiving them again. 

I love working with NASA. 

I have found the videos to be informative and helpful in addressing National Standards. 

I usually use the 60-minute tapes in two parts, over two or three days. 

We loved the SCI fdes program, but the videos did not capture the attention of my students. They 
loved everything else and really looked forward to using the material. The videos they found boring 
and so we did not view them. We did however use everything else and will do so again this year. 

Very good programs, hope to see more coming in the near future! 

I wish our PBS affiliate carried these programs. It would make copying them easier for us! 

good to use as introduction of concepts 

For some of the programs I did not receive enough advance notice to record the program and use it in 
my classes 

Our PBS doesn’t show NASA programs. I called the nearest resource center to get tape copies and was 
told to drive to Cape Kennedy to make my copies. I sent them some blank tapes and they copied the 
requested programs and mailed them back to me but they said they would not be able to do that 
anymore. It is a 3 hour drive from Jacksonville to Cape Kennedy resource center. I talked to our 
administration and they gave me a limited amount to purchase some programs. The price is veiy 
reasonable but we are a private school and our budget is limited. If I mailed blank tapes to another 
resource center, do you think they would copy the programs for me instead of having to purchase 
them? 

It was easier when the handouts were mailed, as opposed to having to down load them. You are doing 
a great job. 
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The responses below were given as “Other” comments concerning the lesson guides for the 2003-2004 
NASA SCIence Files ™ programs . 


I did have some difficulty with some of the directions in some of the activities; however, I presented 
the activity and allowed my students to help determine how to get the experiment to work. In the end, 
it ended up being a good learning experience for all of us! 

Again, downloading is difficult with the service that is provided by our county. 

A a home schooler, many of the experiments we were not able to do. It would be nice if there were a 
couple of projects that used simpler props. 

Some of the topics were too detailed for my second graders. Perhaps you could produce a scaled down 
version for lower elementary. 

None 

Requirements by the district & state to use the selected curriculum does not allow time to insert 
anything into my day.. .no matter how good it is. 

Please mail to persons who request them. Too much to print or download. Also they are not available 
to download as the program airs. This poses a problem. 

Super resource! 

I at times had difficulty getting the specific guide to a program. 

I used them even without the video segments. 

The guides were wonderful for experiments. 

I would like a hard copy to be made available upon request prior to the video segments as NASA did 
previously with the CONNECT Series as my district has technical problems often or vims scares and 
we do not have access to our hardware. 

Please. Lesson in Spanish 

Excellent materials. 

I had a hard time finding some of the project activity sheets. 

It takes far too many “clicks” to find the things you need on the NASA web sites, especially from other 
NASA web sites 

yes 

I haven’t had a chance to go through them, but I’d like to use the lesson guides at some point in the 
future. 

I like having them available on hard copy year after year. 

These were easy to understand and to use in the classroom. 

I would prefer them to be mailed. I had trouble downloading from school system computer so I missed 
opportunities to use them for some lessons. 
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The responses below were given as “Other" comments concerning the Problem Based Learning 
Activities for the 2003-2004 NASA Science Files ™ programs . 


Sometimes I had to explain a little more than was in the activity; perhaps a simpler version could be 
developed. 

My students enjoyed the activities veiy much. Time was a limiting factor. 

My class truly enjoyed the Create a Martian Life Form activities online. We would love more of these 
projects. 

I only recently got a computer so I was unable to check out the PBL activities. 

Some were a little difficult for my students’ abilities 
Please PBL activities in Spanish 

I would like to see more activities for students with learning disabilities. 

I did find these to be a bit difficult for my students to understand, but when it was possible for them to 
work in teams, they were better able to use them. 

Thank you for them. 

I was not able to use the programs last year but will be using them this year in the lab. 
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